This invention relates to the art of data centers, and in particular to data processing centers in which banks of servers or other equipment are housed in a protected environment. Specific aspects of the invention relate to housings or cabinets for electronic equipment for use in both controlled environments (e.g. computer/data rooms) and non-controlled environments (e.g. ordinary offices, factories, external sites etc.). Although currently intended for housing electronic equipment, the invention is not limited to this use and instead can be utilized with respect to any equipment for which forced air cooling is useful.
Data centers are important business facilities which aim to provide protected environments for housing electronic equipment, such as computer and telecommunications systems, for a wide range of applications. With ever-increasing numbers of individuals and businesses relying on the internet, hence giving rise to e-facilities such as application service providers, internet service providers, network operation centers, and co-location and web-hosting sites, data centers are becoming busier and more common. In particular, the growth of the internet has resulted in unprecedented levels of server-based computing. Providers have found that many of their network infrastructure and web applications work best on dedicated servers.
While this invention will be described in the context of internet-related equipment, the invention has broader applicability. References to data centers in this specification are therefore to be construed broadly to encompass installations that do not necessarily relate to the. internet, such as those involving telecommunications, or any other equipment assembly using forced air cooling.
Computer and telecommunications systems are commonly gathered into data centers because their sensitive electronics require protection from hazards in the surrounding environment such as air-borne dust, spillages and fluctuations in temperature and humidity, as well as from the ever-present danger of power disturbances such as outages, surges and spikes. Flooding and fire monitoring/suppression systems are also required. It makes commercial sense that these protective facilities are shared between users or tenants of data centers, although some data centers are devoted to a single corporate user.
The equipment housed within data centers can be very valuable, but the data stored by that equipment is potentially of much greater value, as is the cost of downtime should the equipment fail. Consequently, the operators of data centers assume a great responsibility for ensuring the protection and continuous fault-free running of the equipment that they house. Tenants can be expected to claim substantial damages if those objectives are not met.
In recent years, items of electronic equipment such as servers have shrunk in size to the extent of being suitable for rack mounting. Now, therefore, servers in a data center are usually housed in equipment racks or cabinets of a generally standard size and shape enabling servers and their supporting equipment to be housed in a modular, interchangeable fashion. Racks or cabinets are typically supported on raised floors beneath which complex cable networks for electrical interconnection for both power supply and system communication can be laid while allowing access for maintenance and re-routing.
An equipment rack is an open frame with a system of uprights having holes spaced at set modular centers, which are referred to as unit spacings or, for short, U's, 1U represents a vertical spacing of 1.75 inches (44.45 mm). The width between the uprights (a unit spacing width) usually conforms to a standard of either 19 inches (483 mm) or 27 inches (675 mm). Electronic equipment is typically manufactured in a chassis form for rack mounting in accordance with these standard modules, although non-standard sizes are of course possible for specific applications.
An equipment cabinet is essentially a rack as described above but mounted inside an enclosure. The cabinet has access doors at front and rear to allow maintenance access to the equipment within and provides a degree of physical security to the equipment. A typical so-called ‘standard cabinet’ would have an external width of either 600 mm or 800 mm, an external depth of either 800 mm or 900 mm, and an external height of at least 2100 mm. Such a cabinet would be capable of accommodating a stack of 42 to 45 1U units and so would be termed a 42U or 45U cabinet as appropriate.
Units mounted within a rack or cabinet need not necessarily be server units: for example, uninterruptible power supply (UPS) units are often installed to maintain and smooth power supply to other units.
The access doors of a cabinet may be solid, glazed, perforated or a combination of these, and will usually be lockable by means of keys or digital keypads. More sophisticated locks relying upon scanners such as thumbprint or palm readers are also possible.
Equipment racks or cabinets are typically arranged in rows within a so-called technical space in a data center with an aisle space between them of approximately 1200 mm and sometimes as small as 900 mm. This aisle space, also known in the art as ‘white space’, affords access to technical personnel for the purposes of maintenance, monitoring, installation and so on.
As mentioned above, some data centers may be shared by several tenants who house equipment there and so require access to the center. While this raises security concerns, these concerns may be partially overcome by restricted personnel access to the technical space. However, the more access takes place, the more difficult it becomes to maintain a closed environment in which temperature, humidity and ingress of dust or other contaminants can be controlled. For example, to achieve a clean environment IP (ingress protection) rating under British and European standards, a sealed filtered system is required which is difficult when the sealed system is an entire room. Also, a multi-tenant facility also increases the chances of accidental damage to equipment, such as impact damage or spillage of liquids.
Smoke and fire detectors and fire fighting capabilities are important features of data centers. Early fire detection and efficient fire suppressant systems are vital for minimizing equipment damage, data loss, system downtime and service interruption in the event of an actual or impending fire.
To minimize equipment damage, many users and operators prefer inert gases to water sprinkler systems for dousing fires. However, in recent years, some operators have omitted gas protection due to the high capital cost of the system and the high cost of recharging the system once discharged. Many gases are also environmentally unsound. In known environmentally conditioned data center systems, the gas or water system normally discharges into the technical space as a whole and the entire room is closed off, and the equipment within shut down, while the particular server or other equipment unit at fault is detected and removed.
Equipment manufacturers and industry standards specify tight tolerances for environmental conditions to ensure optimal performance of the equipment. For example, relatively small but sudden fluctuations from the recommended operating temperature (e.g. at a rate of temperature change of as little as 10° C. per hour) can cause thermal shock and damage delicate circuits. High humidity can corrode switching circuitry causing malfunctions and equipment failures, whereas low humidity can promote static electricity that interferes with proper equipment operation.
The environmental conditions of a data center are largely determined by a combination of the equipment heat load in the room and the temperature and humidity loads resulting from infiltration of outside air. Other load factors include people working within the technical space, who introduce heat and humidity, and lighting of the technical space, which introduces radiant and convective heat. However, the dominant challenge in environmental control of the technical space is the generation of heat by the electronic equipment housed within.
The heat generated by electronic equipment is related to the power consumed by that equipment. New designs of electronic equipment which are more compact than previous models tend to have higher power consumption and therefore a greater heat output. In particular, the desire for compactness has been driven by the commercial need to fit as many servers as possible into existing data centers. Smaller servers and a denser population of servers are required to return as much revenue as possible per square meter of rackable area within the data center.
To this end, servers have been designed that fit into 1U of space; these ‘1U servers’ are also referred to as ‘high density’ servers. Such servers are relatively heavy and have a high heat output of up to 1000 BTU's per hour (293 W) per server, the level of which depends on the server configuration in terms of number of processors, hard drives etc., and the software type and data being processed. So, while racks or cabinets can in theory be filled with such high density equipment, in reality, overheating may occur in that event. Servers present a considerably greater challenge in this respect than other equipment apt to be rack-mounted, such as UPS units.
Overheating has become a major issue since high density server deployment began. As recently as three years ago, typical electrical loads were between 300-400 W/m2 of rackable space within the technical area but today 1200 W/m2 is the average with some installations being as high as 2000 W/m2. This increase in power consumption is reflected in heat output within the technical space, which in turn adds to the cooling load and hence to the overall power requirement of the data center. Indeed, the overall power requirement has almost doubled over the three-year period. This presents another challenge in terms of inadequate power supply.
Those designing cooling systems for data environments are faced with not just the problem of ever greater cooling load requirements, but with predicting the size of the load to be allowed for at any given point of time. Moore's Law predicts the doubling of semiconductor performance every eighteen months. If Moore's Law continues to hold (and it is anticipated that it will, at least through 2005), there will continue to be a dramatic and continuing increase in product power densities, coupled with the design of smaller devices which is now being referred to as the problem of ‘compaction’.
With compaction comes an increasing amount of cabling to make the connections to the greater number of smaller devices housed in a rack. New category cables such as ‘Cat 6’ are larger in diameter than those of earlier generations, which both restricts airflow within the rack and adds to the total rack weight. The electronic equipment itself now tends to be provided with dual-corded power supplies, or triple-corded in some cases. The power supplies, often ‘hot pluggable’, add to the device weight and therefore to the overall rack weight. In fact, some racks have to be braced as they near their maximum slenderness ratio.
The risk of overheating means that cabinets are very often left unfilled: they are not fully populated with servers or other equipment, meaning that some available levels remain unused. This is to the detriment of efficient space utilization within the data center and, ultimately, increases the cost of housing each server because fewer servers or fewer tenants share the infrastructure costs of the data center.
In existing data centers, temperature regulation is commonly achieved by close control room air conditioning units (also known as computer room air conditioning or CRAC units) within the technical space. An example of such a conventionally cooled data center is shown in FIG. 1.
The conventional computer/data room cooling technique illustrated in FIG. 1 is the typical approach to cooling electronic equipment currently employed. Within an enclosed room 1 defined by a room floor 5, side walls 3 and a ceiling 4, a suspended raised floor 2 is installed at a pre-determined height above the base floor 5. The suspended floor 2 and the base floor 5 together constitute a double floor structure defining a free space or floor void 6 which is used as an air passageway and often as a technical service zone for power and data cables. The raised floor structure 2 comprises a plurality of panels which permit access to the floor void 6 from above.
Sometimes a suspended ceiling 7 having a plurality of ceiling panels is provided below the base ceiling 4. The suspended ceiling 7 and the base ceiling 4 combine to form a double ceiling structure defining a free space or ceiling void 8 which is used as a technical service zone for cables, light fittings etc.
Open racks 9 or rack cabinets, into which electronic devices may be fitted, are disposed on the raised floor 2 within the ‘technical space’ defined by the room 1. Power and data cables for the racks 9 may run through the floor void 6 of the raised floor, the ceiling void 8 of the ceiling structure, the room space 1 over the floor or beneath the ceiling, or any combination of these. When cables are run at a high level in the room space 1, the suspended ceiling 7 is often omitted. Often, so-called static cables should be located in the floor void, these cables being mainly heavy copper cable such as for power supplies, and control data cables, all of which may be expected to remain in situ for extended periods. Conversely, more fragile or frequently-moved cables should be located at a high level within the technical space or in the ceiling void 8 where they can be concealed by removable panels and/or supported by suitable supports (not shown). These cables may include data cables such as fiber, twisted-pair and coaxial cables. Such cables are relatively vulnerable to damage, as may for example be caused by maintenance engineers walking on exposed cables when the floor panels are lifted. Positioning such cables at a high level reduces the risk of damage and eases access to them for installation, maintenance and re-routing.
A rack 9 comprises a vertical framework of rails provided with suitable mounting holes to appropriate industry standards (e.g. EIA-310-D), to accept electronic equipment. The rack 9 is either open to the room space 1 or is mounted inside an enclosure to form a rack cabinet 11 which has various air inlets and air outlets allowing cooling airflow to reach the electronic equipment 12 and to carry heat away.
Electronic equipment most commonly ventilates front to back, that is, air flows through ventilation holes in the front of the equipment casing and exhausts through holes in the rear of the casing. Small fans, usually in the rear of the casing but in some cases in the front, assist this through airflow. The heat from the electronic components within the casing is dissipated by convection or individual fan assistance into this through airflow, thus effecting cooling of the equipment. There are some items of equipment that ventilate bottom to top, or side to side, but the principle remains the same.
The rack cabinet 11 will most often be provided with ventilation slots or perforations in the front and rear doors to provide for through airflow. Many rack cabinets will also have a ventilation inlet within a cabinet bottom plate 13 and another in a cabinet top plate 14 to avoid a concentrated heat build-up in the top of the cabinet due to the so-called ‘stack effect’. Either of these two openings or in some cases both may be augmented by fans 15, preferably in multiple arrays to provide for redundancy.
The raised floor system 2 comprises a plurality of floor panels, some of which are solid and some of which are perforated or of a grille-type construction 2b permitting airflow through them from the floor void 6 to the room space 1. The suspended floor panels are supported on pedestals 2a which are fixed to the base floor 5 by screws and adhesive on a regular modular grid. The airflow from the perforated panels or grilles 2b flows out into the room space 1 and is drawn through the fronts of the equipment cabinets 11 into, through and between the units of electronic equipment within.
Sometimes dampers will be fitted to the floor grilles 2b to allow adjustment to the airflow and manual balancing of the room loads.
At the perimeter of the room 1, a plurality of close control computer room air conditioning units 16 (CRAC units) receive air flowing perpendicularly to the rows of rack cabinets 11. In large rooms, some of these CRAC units 16 may also be provided within the room space away from its periphery to overcome distance limitations upon the effectiveness of the units 16. Also, CRAC units 16 may sometimes be positioned outside a room and connected to it through appropriate openings in the perimeter wall of the room 1.
Each CRAC unit 16 comprises a heat exchanger or coil 17 and a fan 18. Exhaust air from the rack cabinets 11, mixed with room air, is drawn into an inlet 20 of the CRAC unit 16, across the cooling coil 17 and through the fan 18 and out an outlet 19 into the floor void 6. There are various types of CRAC units each of which rejects the room heat absorbed in different ways, namely chilled water units, direct expansion water-cooled units, direct expansion air-cooled units, direct expansion glycol-cooled units and others. However they typically share the same principle, which is that the absorbed heat is conveyed to a suitable point outside the room 1 where it is dissipated to atmosphere using conventional air conditioning technology. Thus, the unit is connected to a central plant via a cooling circuit which may consist of distilled water or other refrigerant. The cooling circuit dissipates heat to the atmosphere outside the technical space via heat exchangers such as cooling towers or external radiators (dry coolers).
The CRAC units 16 described and illustrated are referred to as ‘downflow’ units reflecting the generally downward airflow within them in use, but ‘upflow’ units are also available. Upflow CRAC units are used, for example, where there is insufficient room height for a suspended raised floor or where the equipment servicing philosophy is such that services are run overhead, thus obviating a raised floor. Either way, the principle is similar except that in the case of an upflow CRAC unit, the air inlet is at the front bottom of the unit and room air passes up through the unit before being expelled at the top where it moves out and down in front of the rack cabinets 11 before being drawn through the electronic equipment as in the previous scenario.
Ambient room air is typically at a temperature of 22° C.±2° C. with relative humidity of 50%±5%. The CRAC units 16 supply chilled air to the floor void 6 at approximately 13° C. which is drawn into the rack cabinets 11 by either convection from the airflow from the perforated panels/grilles 2b or by the effect of the cabinet fans. As the chilled air passes through and around the electronic equipment units and is heated, it exhausts out into the room space 1 at approximately 35° C. The heated air then mixes with the conditioned ambient air of the room which is at a temperature of approximately 22° C., and the mixed air then returns to the CRAC units 16 at a temperature of approximately 24° C.
Other perforated tiles 2b are positioned throughout the room 1 to provide air supply to other heat-generating equipment and to maintain an appropriate ambient environment in the room 1. Some rooms are laid out on the principle of ‘hot and cold aisles’, in which rows of cabinets 11 are arranged so that their fronts face each other across a ‘cold’ aisle, from which cold air is drawn into the opposed cabinet fronts, and their backs face each other across a ‘hot’ aisle, into which warm air is exhausted from the opposed cabinet backs. Perforated panels or grilles 2b are only placed in the cold aisles (other than those serving other pieces of equipment elsewhere in the room), thus seeking to ensure the maximum cooling effect by not mixing cold and hot airflows in the same aisles.
The large floor-mounted CRAC units distributed around the perimeter and sometimes in the center of the technical space take up floor space and room volume that could otherwise be devoted to racks or cabinets. This ultimately costs money by loss of potential revenue generation. However, the alternative of air conditioning vents positioned in the ceiling or in walls close to the ceiling, as commonly found in offices, is not suitable for a data center. This is because as heated air rises from the server racks or cabinets, it mixes with the cooler air blowing down from the air conditioning vents leading to condensation and formation of water droplets which can damage computer equipment. Hence, it is preferred that air conditioning vents are located below the server racks so that the natural airflow is not disrupted.
As can be seen in FIG. 1, the cold air cooled by the condenser is forced out 5 underneath the CRAC unit 16 below a raised floor 2 on which the CRAC unit 16 is mounted. The raised floor 2 acts as a plenum under positive pressure. Some of the cold air is forced up through cable/ventilation holes in the base of the equipment cabinets 11 mounted on the floor, while some rises through floor ventilation grilles 2b around the cabinets fitted with control dampers. Thus, the total volume of the air in the technical space is conditioned.
As already mentioned, some of the conditioned air in the technical space is drawn through the equipment cabinet 11, for example via perforated doors, by small fans within the servers themselves. This air flows through and around the heat-generating electronic components within the servers and exhausts as hot air at the rear of the server into the cabinet 11. In some cases, however, solid doors are used at the rear of the cabinet 11 and hot air is expelled at the top of the cabinet 11 through an opening, sometimes assisted by additional fans 15 to avoid a concentrated heat build-up in the top of the cabinet 11 due to the ‘stack effect’. The hot air then returns to the room 1 where it mixes with the room air and is eventually recirculated through the CRAC unit 16 from which its heat is ultimately rejected to the atmosphere via suitable heat transfer means as aforesaid.
With a general environmental controlling device as described above, all the air within the technical space is being continuously treated. Unsurprisingly, the energy demands associated with such an approach represent a significant cost factor. Also, the cooling of individual servers relies heavily upon their internal fans and there may be no attempt to ensure that each server receives its necessary share of conditioned air. Instead, conditioned air may be introduced into the cabinets 11 by various imprecise means that can give rise to conflicting airflows.
Once in a cabinet, conditioned air is left to flow within the cabinet in a way that depends upon the disposition of equipment within the cabinet. So, for example, a server might receive inadequate cooling because adjacent servers nearer the air intake take a disproportionate amount of conditioned air. Similarly, conditioned air might bypass a server by following a path of less resistance, for example through an adjacent empty equipment bay within the cabinet 11. Also, if a server fan should fail, that server will almost inevitably overheat.
The floor void 6, when used for delivery of the cooling supply air, is often assumed to consist of an even mass of pressurized air delivered from a number of CRAC units 16, arranged around the perimeter and possibly the interior of the technical space. The reality is that the floor void 6 contains a plurality of independent airflow plumes emanating from each CRAC unit 16, segregated by boundary layers. Each of these airflow plumes varies in size due to other factors which affects the amount of cooling which can be provided to the technical space.
A factor in airflow plume development is the static pressure within the floor void 6. Assuming initial design is correct, a lack of static pressure may arise from poorly-managed floor openings and/or from close-coupled rack cabinets with additional fans. Specifically, cut-outs for cable entry below cabinets and elsewhere within the room, together with excessive perforated floor panels or grilles, causes overcooling, loss of static pressure and wasted capacity. High-pressure areas of the floor are overcooled while low-pressure areas overheat because a loss of static pressure reduces the size of a plume and hence the volume of room space that that plume is able to cool.
To mitigate this effect, hole cut-outs should be sealed around cables and the floor grilles 2b should be adjusted to deliver an accurate amount of airflow to each cabinet 11. However, in practice, poorly-fitting floor panels or, more usually, floor panels that have been lifted and replaced badly can result in substantial leakage of cooled airflow from the floor void. If the floor void is used for containing cabling then engineers installing cables typically remove a complete row of floor panels and/or stringers rather than leave occasional panels (typically every fifth panel) in place to keep the floor ‘locked-in’, with the result that the panels shift across the whole floor in a process called ‘fish-tailing’, causing gaps to open up.
By way of illustration, site investigations have verified cases where only 31% of the total cooling airflow was being distributed through ‘engineered’ openings, with the remaining 69% circulating out of cable cuts, gaps around equipment and openings from rack cabinets. The cold air escaping in this way returns to the CRAC units 16 without effectively transferring heat from the equipment. This cold ‘return’ or ‘bypass’ air disrupts the heat transfer that could have been available to overloaded air conditioners, in such cases reducing the effectiveness of the CRAC units 16 to just 52% of their capacity.
The act of installing cabling within the floor void 6 further restricts airflow through the floor void 6. This is a degenerative effect, as the rack cabinets 11 are populated over time and a potentially greater heat load created, the additional associated cabling further restricts the airway supplying the cooling airflow through the floor void. The combination of new cabling technology, in which cables tend to be of larger diameter, together with electronic equipment ‘compaction’ results in more densely occupied equipment spaces, connected by increasing amounts of cable.
While it is correct in principle to attempt to reduce the inlet air temperature entering the rack cabinets 11 by increasing the airflow rate through the perforated floor panels, this is an oversimplification. If the velocity is too high, then the airflow can overshoot a rack cabinet 11, tumbling into the hot aisle at the rear of the rack cabinet 11. This wastes supplied chilled air and, by mixing with the exhausted heated air from the rack cabinet 11, lowers the temperature of the exhausted air and therefore reduces the capacity of the installed air conditioners.
Moreover, unless carefully engineered, increased air velocity can create a ‘wind tunnel’ under a raised floor. The increased air velocity reduces potential static pressure, and may be so high that sufficient static pressure to deliver adequate volumes of cooling air up through the floor may not develop for 9 m to 12 m beyond the point of fan discharge from the CRAC unit 16. This results in insufficient static pressure close to the CRAC unit 16 to move the available cooling air up through the floor grilles 2b. Worse still, in some cases, heated room air is actually sucked down into the floor void through the grilles 2b, reducing the cooling capacity of the cooling airflow and creating ‘hot spots’.
The objective of hot and cold aisles is to separate the source of cooling air from the hot air discharge which returns to the CRAC unit inlet. However, in practice, such physical separation is difficult to achieve in an open room environment particularly where high heat loads are concerned. Close-coupled rack cabinets each provided with extract fans create a ‘chimney’ effect to pull air from the raised floor up through the cabinet and the equipment therein. Too often, however, these fans exhaust more air than the CRAC units 16 can deliver, thereby overwhelming their cooling capacity. Also, excessive suction created by these rack cabinets 11 causes heated air from the room 1 to be pulled into the floor void 6 and then up into the rack cabinets 11. There is just not enough cold air from the CRAC units 16 to satisfy the overwhelming quantity of air exhausted by the rack cabinet fans. Some studies have revealed that bypass air problems typically limit CRAC units to less than 35% of their ‘nameplate’ rating. In ‘hosted’ environments, close-coupled rack cabinets have earned the title ‘bad neighbor devices’, in that they take more than their share of the available cooling airflow.
Orientation of the CRAC units 16 in relation to the rows of rack cabinets 111 is not significant at low loads with a clear floor void 6. However, as cooling loads or cabling and other sub-floor obstructions increase, their orientation becomes significant. Ideally, the CRAC units 16 should be orientated such that their airflow is perpendicular to the rows of rack cabinets 11, as placing them parallel to the rows of rack cabinets 11 will tend to create hot spots. This orientation of the CRAC units 16 may place an ultimate limit on total cooling capacity. For example, two out of four wall surfaces in a room may be available for locating the CRAC units 16, the longest of which are approximately 2.4 m wide, with a capacity of approximately 100 kW. Placing more CRAC units 16 on the other two walls will almost certainly result in disrupted airflow/turbulence. More CRAC units 16 can be added within the body of the technical space (which will typically be the case in wide data rooms) but this inhibits data rack layout flexibility.
Humidity should be maintained at a level that avoids static electricity problems. However, to provide stable humidity, it is not advisable to equip each CRAC unit 16 with a humidifier. Slight drift in humidity sensor calibration may cause a CRAC unit 16 to add humidity while an adjacent CRAC unit 16 is simultaneously trying to dry out the air.
This fails to provide a stable environment and pours significant energy down the condensate drain, increasing risk, maintenance, repair and capital costs. Rather, good practice suggests that a centralized system for humidification should be used, which is usually the make-up air system for the room space. If the chilled water temperatures are too low, this shifts cooling coil performance toward dehumidification and lowers cooling capacity.
Those skilled in the art will appreciate that individual CRAC units cannot share load with their adjacent or opposing partners and in most situations the temperature gradient varies widely due to the variety and capacity of the items of heat-generating equipment as well as their operational state at any given moment in time.
Once cooled air has been delivered through the floor void 6 and into the room space 1, that airflow enters the rack cabinet to cool the equipment housed therein. Conventional rack cabinets have perforated doors front and back to allow through airflow front to back. This through airflow is achieved by the combined action of: air being drawn through by small fans associated with the equipment itself; air being drawn through by fans associated with the cabinet (e.g. mounted at the top, bottom, middle etc.), if fitted; forced convection from the raised floor perforated panels or grilles; and/or forced convection directly into the bottom of the cabinet.
Perforated doors may work satisfactorily at relatively low heat loads but, with high density loads, the doors themselves offer resistance to the desired through flow. While cabinet fans can help to eliminate the hot spots that tend to occur at the top of the rack cabinet 11, care has to be taken in sizing these fans in relation to the through airflow. Tests have shown that the cabinet fans can set up a strong ‘chimney effect’ airflow pulling the air out of the top of the cabinet 11. This primary airflow entrains the room air at its boundary, increasing the mass of moving air while reducing its velocity. This tends to set up secondary circulation and reduce the through flow into the equipment itself.
Mention has already been made of the problem with air from the perforated floor panels or grilles being of such a velocity as to pass over the cabinet 11 and into the hot aisle. Conversely, lack of velocity can result in a cooled air supply stopping less than half way up the cabinet 11 and therefore not reaching equipment at higher levels. This equipment will rely on cooling provided by the room air which is drawn through by the equipment fans, which room air may itself already be heated and of limited cooling capacity.
Forced convection directly through the bottom of the cabinet 11 may result in similar problems to those noted above. However, additionally, if the rack 9 is heavily populated, then the incoming air strikes the base of the first server and may be deflected out of the cabinet 11 through the perforated doors both front and back. This wasted cooled air then mixes with the room air. During tests using standard industry rack cabinets with forced ventilation through the cabinet base full of 1U high density servers, it was found that the temperature gradient at the rear of the rack became inverted with the highest temperatures recorded within 150 mm of the base of the cabinet. This was largely due to the incoming chilled airflow being forced directly out of the cabinet by the lowest servers and back into the room 1.
In general, conventional rack cabinets are rather ‘leaky’ not just externally but also internally: for example, many have gaps between the rack itself and the cabinet enclosure allowing cooled air to bypass the equipment within the cabinet and be wasted.
Currently, good practice dictates that due account is taken of individual cooling requirements when arranging the deployment in a rack cabinet, especially to avoid placing very hot devices below equipment with lighter heat loads. Even where this practice is followed, rising heat will tend to result in an accumulative heat build-up progressively towards the top of the rack cabinet.
The majority of rack cabinets are never full of hot devices, up to 40% percent occupation density being typical. Sometimes relatively high loads are possible within standard cabinets—perhaps up to 5 kW. However, upon examination, this is usually due to the load being created by relatively few devices. For example 5 kW of aggregate cooling load from two items of equipment with plenty of air space between them is very different from 5 kW of aggregate cooling load from a full rack of hot equipment. Also, equipping the server itself affects the resistance to airflow. For example, a server fully equipped with network cards might offer 64 pascals of resistance while an otherwise identical but less equipped server might offer only 20 to 30 pascals of resistance. Further, the loads of adjacent equipment directly impacts cooling capacity.
If one considers that a raised floor is effectively a supply air duct and that the length of a typical data room is considered as the duct width (say 37 m) and a room area of approximately 1000 m2 (27 m×37 m) is taken as an example, then taking the averaged loads across the room:
(a for a floor void height of 600 mm, and a heat density of 2000 W/m2, the duct should be 108.9% of the room length;
(b) increasing the floor void height to 800 mm for the same heat density results in the duct being 81.9% of the room length; and
(c) if the floor void is cabled out, reducing its effective depth to only 300 mm, then the duct should be 218.1% of the room length. Put another way, the maximum achievable heat density is less than 2000 W/m2 (108.9%).
Further if one considers the maximum floor space that can be occupied by active IT hardware (rack cabinets) in the above scenario (typically between 30 and 35 percent allowing for all white space such as service clearance, access aisles etc.) and a rack cabinet footprint of 0.54 m2 (0.6 m×0.9 m), then each cabinet can provide between 3.6 kW and 3 kW of cooling capacity. This is a theoretical capacity with a completely free floor void: more realistically, with cabling in-situ, this figure will drop to between 1.8 kW and 1.5 kW per cabinet.
Using the same scenario, consider the total available cooling load from correctly oriented CRAC units. Assuming access and fire exit doors are allowed for (one per side, 1.2 m wide) then a maximum of 14 CRAC units of 100 kW capacity can be located on each side. Assuming a minimum availability of n+1, then a total of 26 CRAC units are available for a load of 2,600 kW. This equates to 2.6 kW/m2 or 4.68 kW to 4 kW of cooling capacity per cabinet. However, to be able to deliver this capacity, the size of the supply duct would need to be increased by raising the floor void height to 1500 mm of clear space (i.e. above any cabling also within the floor void, which is impractical for most data centers. Even for new purpose-built facilities, this floor depth presents various technical challenges.
Other factors affecting the movement of airflow within a given space are ceiling and wall topography, such as: surface characteristics; type of surface; downstand beams; surface obstructions; pipework; ductwork; services; wall abutment; placement of supply grilles relative to equipment racks; placement of equipment racks relative to each other; other equipment; and wall abutments.
Predicting the effects of these various parameters with any certainty to achieve an optimized room configuration is extremely difficult. Technology such as computational fluid dynamics (CFD) software can assist greatly but this approach is not, as yet, widely adopted within the industry. Also, to be effective, the computational model requires accurate modelling of all the characteristics of the room, the rack cabinets and the heat producing devices. Many manufacturers do not make available the information necessary to undertake this task.
The result is that while some security disadvantages have been overcome by containment devices which contain electrical equipment and are positioned in an environmentally controlled room, many of these existing installations risk overheating the equipment due to unexpectedly dense levels of deployment and poor or inadequate air extraction and ventilation. Indeed, in some data centers, the doors that are supposed to be closed to provide security are instead left open to assist cooling.
Enclosed cabinets with perforated ceilings onto which fan kits can be attached to aid air circulation through the units are also known, but they are ineffectual in high density applications. These systems still rely on environmentally cooled room air and the cooling effect of the fans is negligible. While fans may help to achieve desirably uniform air flow within the cabinet, considerable care should be taken in specifying fan size and capacity because an incorrectly specified fan can inhibit the general flow of air within the cabinet.
Various solutions have been proposed to assist with issues of conventional data center technology, which is now some thirty to forty years old. These break down into two main approaches: room-level solutions and rack-level solutions.
Room-level solutions begin with ‘close coupled’ cabinet systems. Essentially these are developments of the existing pressurized floor technology. The individual cabinets attempt to make better use of the supply air by controlling the amount of air entering each unit with a variety of different types of air inlets or dampers. These dampers are positioned in the base of the cabinet, and on the cabinet front, normally low down to take advantage of the coolest level of room air. Some have pre-engineered openings for cables with sealing brush strips to mitigate the effects of unmanaged and unsealed cable cut-outs. Often, bottom air inlets are provided with small fans to assist airflow into the cabinet enclosure, and in some cases fans are also provided in the top of the cabinet enclosure for the same purpose.
Some variants have recognized the limitation on total cooled air supply from the floor void i.e. between 2 kW/m2 and 3 kW/m2 maximum. These variants attempt to use the room air in addition to that from the floor void to increase the total cooling capacity, which is achieved by manually adjustable opening vents in the cabinet front.
Existing room-level solutions do nothing to address several issues with raised floor technology, which include:
limited total cooling capacity with reasonable floor void heights;
increasing sub-floor obstructions reducing the airway over time;
wasted cooling air through poorly-managed cable cut-outs and leaking floors and cabinets;
lack of or too high a static pressure;
uneven loading on CRAC units with some under-utilization and some overloaded;
problems of bypass air;
uneven temperature gradients across the room space; and
influence of adjacent items of equipment on each other.
Further, in attempting to boost the cooling capacity by using room air, these solutions assume that a background ambient temperature of around 22° C. is available. For the reasons already given, this is often not the case as room air is heated by recirculated exhaust air. Using fans to pull air through a cabinet requires a careful balance and sizing of fans to achieve the desired through flow with the racked equipment. Strong vertical airflows can be set up which remove much of the cooled air before it reaches the equipment and has the effect of throttling-down the through flow desired and being attempted by the small racked equipment fans. The vents and dampers on these systems normally require manual adjustment which is carried out on a trial-and-error basis. As racks are equipped out over time, adjustments are often not made to the damper settings until there is a problem. Perhaps more seriously, close-coupled systems can draw so much air from the floor void that they exceed the capacity of the CRAC units to supply it, and starve other containment systems of cooling in a prime example of the aforementioned ‘bad neighbor devices’.
A different approach is used for ‘spot coolers’ which, in one example, places a heat exchanger on the rear door of the cabinet, complete with a number of fans. The fans pull room air through the cabinet and the equipment racked therein, into the heat exchanger and then exhaust it back into the room at or near room temperature. The airflow is therefore front-to-back through the racked equipment as required by the majority of equipment suppliers.
The heat exchanger coil is connected via a pipework system to a cooling distribution unit located outside the technical space which, by regulating the temperature and flow of the chilled water in relation to changes in room dew point, helps prevent condensation. The cooling distribution unit is connected to the existing chilled water supply of the building. Between twelve and fifteen heat exchangers can be controlled from one cooling distribution unit, giving a cooling capacity of up to 8 kW per cabinet.
This system has been designed as a ‘hot spot cooler’ for retrofit situations. The heat exchanger used, however, intrudes into the hot aisles by 150 mm, thereby reducing the width of each hot aisle by 300 mm if used in adjacent opposed rows.
Due to uneven temperature gradients and complicated airflow patterns already discussed, it is a distinct possibility in many data rooms that the room air drawn into the rack cabinet is above the designed ambient temperature, e.g. 22° C. In that case, the air being exhausted back into the room by the cabinet heat exchanger may also be above the designed ambient temperature. In the case of hot aisles where ambient air is at, say, 34° C. (assuming other containment is exhausting into them—the situation one would expect with a retrofit hot spot solution), there will be some cooling of the air in the hot aisle by mixing of cooler air from the cabinet heat exchanger. This may lead to a phenomenon called ‘static bypass’ which lowers the cooling effect of the CRAC units, creating other hot spots.
While connections are made into the existing building chilled water service, which would not be permitted in some data facilities, this system has the advantage of allowing progressive build-out. Provided that care is exercised in positioning relative to other equipment, it provides a good technical resolution over conventional technology with regard to equipment cooling, albeit limited given the heat loads now being encountered. However, the system is ‘open loop’ and so is still vulnerable due to complicated airflow patterns within a conventional system. Similarly, the equipment in the rack is exposed to the other issues already discussed with conventional open rack cabinet systems, such as dust, moisture, cold smoke damage, security and fire risk.
Moreover, the cooling distribution unit is linked to remote room-mounted temperature and humidity sensors. So, this is a ‘centralized’ control system rather than a rack-specific control system.
The pipeline supply system connecting the cooling distribution unit to the rack heat exchanger is typically single pipe with a mechanical coupling joining the pipe sections. The type of coupling used is a ‘Victaulic style 606’ (trade mark) which provides a very high quality joint. However, such a joint cannot be said to be leakproof, and combined with the use of solenoid valve assemblies in the pipe runs, as valves are a potential source of leaks, the pipe system may have leaks, even if dual-piped which is not a standard option. If a leak is detected, an internal purge system pumps the coolant within the coolant distribution unit to a drain.
Another variant of the ‘open loop’ system is the ‘zero floor space’ model. One embodiment of this approach locates a heat exchanger at high level above the rack cabinets such that cooled air is washed down the fronts of the individual cabinets. This is a similar action to that of the pressurized floor solution, but in the reverse direction. The airflow passes through the racked equipment due to the action of the internal equipment fans and forced convection from the overhead heat exchanger fans. The exhausted heated air is then drawn back up into the overhead heat exchanger to be cooled and the cycle repeats.
The overhead heat exchanger is connected via a pipework system to a cooling distribution unit located outside the technical space and then to the building's chilled water supply. Once again, this system relies on central control using a remote room temperature/humidity sensor.
Each module of the system is 1.83 m×1.8 m in plan area which covers three conventional rack cabinets and weighs 160 kg when filled with coolant. The units are attached to the structural soffit by threaded rod and appropriate anchors, which means that this solution is not, at least primarily, a retrofit option but for new-build situations.
Each module is 0.55 m high and has between 0.6 m and 0.9 m of clear space between the module's bottom face and the top of the rack cabinets. Any suspended ceiling fitted is located at the same level as the cooling module. The spacing between the modules in plan is varied to suit the room load. If the units were butted together edge-to-edge, this would give a notional 6.6 kW per cabinet of cooling. However, this level would not be achieved in reality as the system is effectively ‘open’ and subject to all the same room restrictions as for a pressurized raised floor. With edge-to-edge abutment or substantially so, there is insufficient room to install light fittings or overhead cable management (which is increasingly the preferred option among users), either of which would disrupt the airflow pattern if fitted below the cooling modules.
Placing the units edge-to-edge on their shortest sides (to fit cabinet/aisle widths) and, say, two cabinets apart on their longest side would provide a notional 4 kW per cabinet. The real cooling load delivered to the rack cabinets is likely to be just above that provided by a pressurized raised floor. However there is the advantage that the overhead situation does not have to deal with the floor void and room topography restrictions to airflow inherent in the raised floor design. The supply and return air path from rack cabinet to heat exchanger is relatively short, and floor space is saved for use by revenue-generating equipment. The raised floor can be used for static cabling and possibly dynamic cabling can be run overhead although even with spaced-apart modules, cabling along the line of the rack cabinet is not possible at a high level other than directly on top of the cabinets. Nevertheless the spaces between the cooling modules could be used for cable bridges between rows.
The next category of solution is the closed-loop chilled water group which is sometimes described as ‘air cooled’ systems because only air is used within the rack cabinet itself. However these systems are, in reality, a development of the traditional pressurized raised floor technology, in that they rely on CRAC units to transfer the heat from air to water or refrigerant and then ultimately to the atmosphere via external chillers or water towers.
One particular example of this approach, as disclosed in International Patent Application No. WO 01/62060, seals the cabinet and directs the airflow via a front and rear plenum or manifold. Actually the cabinet construction may not truly be sealed because in practice there may be visible gaps in the carcass construction, although there are sealing gaskets on the doors. The air movement is vertical through the front ‘supply’ plenum, then horizontal through the racked devices and then vertical again through the rear ‘exhaust’ plenum. A variety of fans are used, sometimes located at the top of the exhaust plenum and sometimes also at the bottom of the supply plenum. This helps to control the airflow through the racked equipment.
The bottom of the supply plenum is connected into the raised floor, which effectively forms the supply duct. The top of the exhaust plenum is connected into the suspended ceiling void which effectively forms the return air duct. To achieve the function as discrete ducts, the two voids are segregated by vertical barriers. This arrangement allows for a substantial improvement over the normal pressurized raised floor open room return air scenario. A limited number of rack cabinets are directly connected to individual CRAC units forming a closed loop system, thus making far more efficient use of the cooling air available from the CRAC unit. The supply air is delivered at 13° C. while the return air is expected to be 34° C. to 35° C. This compares with a conventional pressurized floor open room scenario of supply air at 13° C. and return air between 22° C. and 24° C. Thus, it can be seen that the closed loop system has a Δt of 22° C. as opposed to the conventional system Δt of 11° C. The principal advantage put forward for this system is that by doubling it, it is possible to reduce the required airflow of cooled air to the rack cabinets. This in turn means the CRAC unit fan requires 50% less power to drive the airflow and thus substantial energy savings are possible. However, this figure assumes that system losses do not reduce this saving even though it is still proposed to use the raised floor for cabling, and other factors such as leakage through the floor tiles. Similarly, a suspended ceiling is not a largely unobstructed duct normally used for ducted air supplies so, again, resistance to airflow and leakage within and from the ceiling is likely to impact on these figures.
Also, with this approach, in a retrofit situation, it may not be physically possible to install a suspended ceiling due to the amount of overhead service obstructions already existing. Additionally, installing a suspended ceiling in a live data center may not be acceptable i.e. drilling into the structural soffit to fix the suspension hangers and so on.
This system still has stack or chimney effects inherent to all vertical airflow systems, requiring very careful management of the deployment of heat-generating devices. While loads of up to 8 kW of cooling are claimed for this solution, this may be difficult to achieve in practice, even assuming the greater fundamental efficiency due to the higher At. Test figures have apparently been based on equipment nameplate ratings or alternatively using heater bars. Real figures can be a third of nameplate ratings under real running conditions. Also, while the use of heater bars is the most common industry approach to testing, this takes no account of the variation of heat dissipation across electronic devices or their resistance to airflow—typically 20 pascals for a near-empty server or other device and up to 64 pascals for one full of network cards. Fully-ducted systems of this type utilizing discrete ductwork and well-sealed cabinets can achieve cooling loads of up to 12.5 kW. However there are limitations on the depth or length of cabinet rows due to air velocity factors—testing has indicated this to be at around 20 standard cabinets (600 mm wide).
Another issue with of this approach from a user's point of view relates to the lower airflow. Original Equipment Manufacturers (OEMs) design their equipment such that small fans sometimes combined with heat sinks move the heat away from the critical components and up into the through air-stream. Additional fans, sometimes as many as eight, pull air through the device to exhaust the heated air at its rear. An OEM's products can be damaged by too high an airflow; especially, it is possible to ‘windmill’ or cycle the small fans beyond their self-driven revolutions and shorten their life or indeed burn them out prematurely. Overcooling can also prejudice the correct operation of a device. On the other hand, too low an airflow can result in local overheating.
The heat levels across a device are not even, some regions being significantly hotter than others. Each manufacturer has varying inlet temperatures for its equipment. However, in most real data center situations there is a mixture of products from different suppliers, or different models from the same supplier in any given rack. Therefore, from a practical viewpoint, a compromise airflow is provided that covers the spread of inlet temperature requirements and variations experienced across the devices. For this reason, the major OEMs have expressed concern regarding any cooling method which intentionally reduces the airflow significantly. Their preference is to tend towards higher airflows as this is more likely to ensure safe operation, rather than move to lower flows.
As with some of the other systems discussed, the control function on these products is effected centrally. The cabinet enclosure is, as already noted, leaky especially if the enclosure is a single-skin construction, especially if not insulated. Consequently, should it be possible to achieve real cooling loads above 5 kW and perhaps up to 8 kW with this equipment, then there will be an impact on adjacent equipment. In general, other neighbouring hot devices are likely to impact on the environment in a given cabinet. While this system includes door seals, the overall cabinet construction does not appear to meet any recognized standard of ‘sealing’ classification.
The next group of products fall under the generic grouping of ‘sealed closed loop air to water category self-contained’. Put simply, the heat exchanger is contained within the rack cabinet itself. The present invention falls within this category but at least one other example is also currently within the market. This unit, the subject of U.S. Pat. No. 6,506,111 issued Jan. 14, 2003, has a segregated supply and exhaust airflow system comprising two plenums—one at the front of the racked equipment and the other at the rear. This is in common with other units discussed previously.
The whole unit stands on a plinth which contains the heat exchanger coil and fans. Heated air from the rear of the racked equipment is drawn downwards through the plenum and into the plinth. The fans push the air through the heat exchanger coil and up into the plenum in front of the racked equipment, through the equipment and back into the rear plenum to begin the cycle over again.
To overcome the stack or chimney effect inherent in any vertical airflow system, various distribution devices are incorporated into the front plenum. The first of these comprises a fiat plate containing a plurality of pre-formed regular apertures.
These apertures increase in number from the bottom to the top of the plenum and thus allow the air to flow through them into the racked equipment. The pattern of the holes can be varied to suit the load with the intention of delivering ‘approximately’ equal airflow to all levels of the racked equipment. It is also possible to have louvres fitted to the apparatus to provide further airflow adjustment, presumably in a manual operation.
Another distribution device option comprises a solid panel with a raised side along its two long edges and a curved roll-over top mounted on the back of the front door of the rack. The panel is tapered in its depth such that it reduces the cross sectional area of the plenum progressively from its bottom to its top. This again is designed to provide even airflow through the racked equipment, in a manner similar to the progressively-reducing section found on any run of heating, ventilating and air-conditioning (HVAC) ductwork. Further it is proposed that the same device can be fitted in the rear exhaust plenum or in both plenums. How successful this system is in providing even airflow across all racked equipment is not known, but it would seem that the current levels of cabling required might be obstructed by these devices in high-density applications.
Multiple fans are provided for redundancy although it appears that it is necessary to ithdraw the complete fan tray to replace a failed fan, with the consequence that airflow is interrupted while this takes place. With high-density applications, even a short period while a fan is swapped out could have serious implications for the racked equipment. Similarly the heat exchanger coil is described as either single, which is not resilient to the minimum requirement of most data centers for ‘n+1’ unless hot swappable, or multiple. The multiple option would provide resilience although if it is necessary to pull the two coils out together to swap out the defective one there does not seem to be any point in a multiple coil arrangement. Perhaps this is the reason that the production units are only equipped with a single coil.
The system of U.S. Pat. No. 6,506,111 has the benefit of removing heat from the racked devices close to where it is generated, and of directing the airflow. This also allows relatively high heat loads to be dealt with—currently up to 10 kW per cabinet is claimed. A raised data room floor is not necessary and the position of the heat exchanger means that small footprint dimensions are achieved, albeit with a loss in rack height (the current variant is 40U). The items containing coolant are located low down in the plinth reducing potential damage from leaks, although no means of leak containment seem to be provided. Although described as a sealed system, this refers to ‘close fitting’ panels' (single skin non-insulated) and not a recognized seal standard or rating—thus the cabinet can only be used in data room environments and is subject to dust, cold smoke, water etc. penetration. External chillers and interconnecting pipework are required.
The final group of known products are sealed closed loop air to refrigerant systems which exhaust the extracted heat into the surrounding space. These systems are substantially autonomous, but most suitable for use in low (or at least not high) density environments where the hot exhaust air will not add to problems for other equipment. However, although some of these products are sealed to a recognized standard, they do not have any means for safeguarding the internal environment where the external environment is not benign, such as conditions of high humidity, partial pressure problems and so on. Some models have the self-contained package unit mounted externally on top, or on the sides of the cabinet. Other variants have it located in the bottom of the rackable area. Cooling capacity tends to be limited with these products, ranging from 1.5 kW up to 4 kW. A condensate drain is required with these products; excessive opening of doors or poor seals can cause continuous drainage of condensate. The room in which these products are located needs to have adequate air circulation to ensure the heat exhaust is rejected to avoid the cabinet overheating.
Examples of such cabinets are currently sold by Liebert Corporation under the name ‘Foundation’ and by Stulz GmbH under the name ‘CT Cooling Rack’. All trade marks are acknowledged.
Liebert's ‘Foundation’ is aimed at small offices rather than data centers. Essentially, it is an enclosed cabinet, which may be sealed, with locks on the outside of the cabinet to prevent tampering. An internal rack-mounted UPS is an option. Various cooling modules can be employed, for example an internal rack-mounted or external top-mounted ‘environmental control module’ that cools equipment within the cabinet by using ambient air to remove heat from the inside of the cabinet through an air-cooled condenser. This of course takes some of the space that could otherwise be devoted to servers, if their heat generation problems could be overcome. Warm air is exhausted near the bottom of the unit.
Other cooling options are a fan that can be mounted inside the cabinet to promote air circulation in the cabinet, and a back-up cooling module which responds to excessive internal temperature by circulating filtered ambient air through the cabinet. Another cooling option is a ceiling-mounted fan for ventilating a confined space outside the cabinet, heated by warm air from the cabinet. Units within the rack cannot be upgraded without shutting down the entire unit.
The Stulz ‘CT Cooling Rack’ is a cooling system for electronic enclosures that can be fitted onto existing cabinets, and is mainly directed to the PABX market in telecommunications. The cooling system is also available with a cabinet comprising three sides and a glass door, with the cooling unit situated on top of the cabinet. Air inside the cabinet is cooled by ambient air that is drawn through a heat exchanger in’ the cooling unit and is then circulated within the cabinet. Again, an internal rack-mounted UPS is an option.
Neither of the Liebert or Stulz products is able to achieve the degree of cooling required by a fully-filled 42U-plus cabinet in a large data center. Also, while their localized cooling provisions go some way toward reducing the contamination and inefficiency issues of whole-room cooling, they still have inefficient and ill30 defined airflow within the cabinet. For example, the top-down flow of cold air from top-mounted cooling units goes against the natural upward flow of warm air, and risks condensation problems as moisture in the rising warm air meets the cold downward flow. Moreover, there is still a risk that some servers will receive less cooling air than they should, and that failure of a server's internal fan will result in overheating.
Returning to the power supply issue mentioned briefly, above, redundant engineering is often built into data centers when they are first established so as to allow for future expansion. This inevitably results in waste of money and resources if the data center does not quickly reach its expected capacity. Conversely, if the data center quickly exceeds its expected ‘capacity, there is a lengthy lead-time because additional power allocation after initial request from a data center tends to be extremely slow. This is a limiting factor upon the natural growth of the data center.
The result is that tenants usually request more power than they need initially, leading to a scalability problem because power infrastructure needs to be installed for the entire data center from the outset day even if there are only a few tenants at that stage. It would be preferable if there was increased flexibility of adding power to a data center at short notice so that tenants would only need to request extra power as and when required.
Thus, correct sizing of appropriate technology is extremely important. If too much site infrastructure capacity is installed, then those making the investment recommendations will be criticized for the resulting low site-equipment utilization and poor efficiency. However if too little capacity is installed, a company's IT strategy may be constrained, or new services may have to be outsourced because there is no space with sufficient site infrastructure capacity to do the work internally.
Summarizing all of the above, those skilled in the art know that thermal characteristics and airflow movement within a typical data room environment are extremely complicated and, to a large extent, full of uncertainty. As cooling loads increase, this becomes more critical. Conventional cooling solutions can cope up to, say, 2 kW to 3 kW per cabinet, provided that cabling and other requirements are modest. Above this level, it becomes necessary to either spread equipment out widely, which may not be practical or cost effective, or to place restrictive limits on the number of hot devices that can be deployed within a rack. It will be recalled in this respect that a typical maximum deployment density is just 40% of rack space. Currently, such limits are often forced on users due to the action of thermal triggers within the electronic equipment.
It is against this background that the invention has been devised.